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ABSTRACT 

Field-tested procedures to collect followup data on 
former students of vocational education programs are described in the 
document. The procedures presented are designed to facilitate the 
collection, analysis, and reporting of information on jobs obtained 
and their relationship to training programs, and the income,*' 
geographic mobility, and the satisfaction of former :^tudents. The 
questionnaire data collection procedures, and examples of possible 
output formats are presented, vith eight figures and 22 tables 
supplementing the discussion. Field*tested procedures included 
identification of the population, file preparation, the 
questionnaire, pre^orientation of the population, mailing sequence, 
sequence of events, and interpretation of data. Advantages and 
disadvantages of collecting followup data from teachers as opposed to 
students are discussed. The development phase of a follovup of former 
vocational students in Kentucky includes information on the 
development of the instrument, data collection, and development of a 
software package for processing and analyzing the data» This, along 
vith examples of the kinds of output and interpretation which may be 
generated using these procedures, make up the appendix. (LH) 
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PREFACE 



Improved methods for evaluating impact and cost-effectiveness 
continue to be a matter of widespread interest. The prime sources 
of data for such evaluations are former students themselves. 
However, obtaining data in this way is often a difficult and dis- 
couraging undertaking. Such unrewarding experiences can result 
from a variety of causes including inadequate instrument (ques- 
tionnaire) design, and inadequately planned and executed procedures 
for data collection. 

This report is concerned with obtaining and using student 
follow-up data as a most critical requirement for evaluating 
impact of vocational programs. A detailed guide is provided, 
including questionnaire, collection procedures, and examples of 
possible output formats. 

Special recognition is due the author, Krishan K. Paul, 
formerly research and development specialist at The Center, now 
Director of Systems Development, Tennessee Department of Economic 
and Community Development. In addition. The Center is grateful 
for the cooperation of the Kentucky Bureau of Vocational Education. 
Without the active participation and interest of Carl Lamar, 
Director, and members of his staff including Janie Jones, Floyd 
McKinnev, Charles Neel, and Billy Vice, the accomplishments re- 
ported here would not have been possible. 



Robert E. Taylor 
Director 

The Center for Vocational 
Education 
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CHAPTER I: INTRODUCTION 



In any information system in which planning and evaluation 
of vocational education are essential parts, follow-up of former 
students occupies a central position. It provides a periodic 
feedback mechanism for evaluation and accountability of vocational 
programs, establishes trends for planning, helps to determine 
manpower supply, and provides an important input for resource 
allocation decisions. It can also provide data input for research 
in cost-effectiveness of vocational education and mobility data 
for manpower demand projections. 

The follow-up procedures presented in this manual are de- 
signed to facilitate the collection, analysis, and reporting of 
information on the post-schooling experiences of graduates and 
dropouts from vocational programs. Of particular interest is 
information on jobs obtained and their relationship to training 
programs, and the income, geographic mobility, and the satisfaction 
of former students. 

The overall objectives of the follow-up procedures are as 
follows : 

1. To provide information for product evaluation (e.g., 
percent of graduates placed on jobs, level of salaries 
and wages earned by graduates, percent who are satisfied 
on the job, etc.) 

2. To provide some process evaluation information on train- 
ing programs .(e.g., how graduates rate their training 
programs and other school facilities, etc.) 

3. To provide comparable information on graduates from 
various vocational and technical training programs within 
a state (private school, adult education, and academic 
education programs) so as to set up norms against which 
future programs can be evaluated 

4. To provide information on placement and geographic 
mobility trends for manpower planning purposes 

5. To provide placement, job satisfaction, and wage infor- 
mation to guidance personnel for counseling purposes 
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6. To provide relevant information for accountability and 
for building up the image of vocational education in the 
conununity 

7. To provide information to fulfill placement-related USOE 
reporting requirements by the states. 

Development work for the initial follow-up was done using 
Kentucky as a test-site state, though inputs were also received 
from Colorado, Oklahoma, and Washington. In the following chapter, 
procedures to collect follow-up data are described which may be 
replicated in other states. Appendix B contains examples of the 
kinds of output and interpretation which may be generated using 
these procedures. 
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CHAPTER II: DATA COLLECTION PROCEDURES 



Former students of vocational education programs in 
every state should be considered the primary source of follow- 
up data, and their teachers as a secondary source. Following 
are some of the arguments which have been offered by various 
researchers in support of utilizing either of these two 
sources • 

Stringent Federal reporting requirements are generally 
cited as one of the reasons for using only teachers for data 
collection. It is argued that the time between the end of 
the school year and the date the report is due in Washington, 
D.C. is not enough to collect and aggregate follow-up data 
obtained directly from former students. There are certain 
other advantages which have been attributed to collecting 
data from teachers instead of contacting former students for 
similar data. These advantages are listed as follows: 

1. High response rate: It is claimed by researchers 
in some states that the response rate from teachers 
tends to be significantly higher than that from ^ 
former students. In most cases, the claimed res- 
ponse is as high as ninety percent. 

2. Complete control of time and schedules: It is 
easier and less time-consuming to collect data 
from teachers than from former students. This 
could be due in part to the fact that teachers 
are subject to considerable state and Board of 
Education contK|)., whereas ex- students are free 
from any such obligations except through emotional 
and social ties to their alma mater. 

3. Lower cost: Data collection from teachers involves 
the use of existing channels of communication 
between the schools and the state office which result 
in considerably less cost than when using the 
students as a source of data collection where 

the only means of communication is via postal 
services . 
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Against these advantages there are a number of dis- 
advantages to collecting follow-up data from teachers. 
Some of these are as follows: 



1. Teacher responses are generally based on the 
students' intentions at the time of graduation 
or leaving school. A notable exception to this 
will be in the case of students from rural schools 
where graduates tend to remain in the area after 
leaving school. In such cases the teacher responses 
may be based on actual knowledge of the students' 
employment status at the time of reporting. In 

the case of urban schools, however, the students 
are more mobile and therefore teachers lose contact 
with them more often. Follow-up information in 
these cases is based on the last known intentions 
of the students. In many cases the information 
is outdated by six to seven months by the time 
teachers normally supply follow-up data to their 
respective supervisors. 

2. In cases where the intentions of graduating students 
are not known to their teachers, there is a great 
temptation to guess the responses which are socially 
acceptable and politically expedient. 

3. There is no known valid method of collecting 
certain detailed but important data from teachers. 
These data include information about wages, job 
satisfaction, mobility, etc. 

4. Even when teachers have information on the place- 
ment of their former students, they are less likely 
to know the relatedness of each job to the prepa- 
ration received in a vocational program then the 
persons actually working on those jobs. Thus, the 
quality of these data collected from teachers would 
be lower than from students. 

5. There is a general complaint from teachers that 
they are required to spend too much time on non- 
teaching functions, such as reporting, leaving less 
time for classroom teaching. This problem is more 
acute in popular vocational programs (such as 

auto mechanics) in urban schools where the class 
size and the number of graduates is generally large. 

6. Since this mode of data collection depends entirely 
on continued personal contacts between teachers and 
their former students, a teacher's departure from 
the scene affects the quantity and quality of data. 
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7. Since one of the important objectives of follow-up 
data collection is program evaluation, it is po- 
tentially very tempting for the teachers to intro- 
duce enough bias into the data to make them "look, 
good." 

Follow-up data collected directly from students, on the 
other hand, are more comprehensive, extensive, and reliable, 
thus offsetting some of the higher cost at which they are 
acquired. Moreover, using techniques developed in Minnesota, 
Wisconsin, California and Pennsylvania, the response rate 
from students can be increased significantly to match the 
higher response rates claimed for follow-up through teachers. 

On balance, although follow-up data collection through 
teachers results in a slight saving in cost and time, it is 
a sound policy to rely on former students of vocational 
programs, the primary source of this data, for follow-up 
and other evaluative information. 

The following procedures have been developed and field- 
tested for collecting such data using a mailed questionnaire 
which has been tested and validated. These procedures are 
recommended to improve quality of data and response rate 
at a reasonable cost. An exemplary sequence of events is 
shown later in this section, though some of the procedures 
do not follow this sequence in the text. This has been done 
primarily for the sake of clarity and logical development. 

Identification of Population 

The first step in any follow-up survey is the identi- 
fication of the population. The population for a follow-up 
may be drawn from any or all of the following groups of 
former vocational students. 

1. Graduates: Graduates include those students who 
completed a vocational program and graduated from 
high school or other post-secondary and adult edu- 
cation institutions. 

2. Completors: This group includes all those who 
completed the vocational program but either left the 
school without graduating or are still enrolled in a 
school or college. 

3. Non-completors : This group includes all those 
former students who enrolled in a vocational pro- 
gram but did not complete and left the school or 
college without graduating. 
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The following two groups o£ former vocational students 
may not be a part of the follow-up population*. 

4. Those who did not complete a vocational program and 
are still enrolled in the same high school or college 
(Relevant information on these students should be 
available through a student accounting system.) 

5. Those who enrolled in a vocational program but, 
before completing, are known to have transferred 

to another school, school district, or to a different 
vocational or non-vocational program. These students 
will be picked up from the school to which they 
transfer. This procedure will help to avoid some of 
the duplication which will otherwise result. 

It may be pointed out here that during initial develop- 
ment of follow-up and student accounting systems, it may be 
expedient to include even groups (4) and (5) above in the 
follow-up population. Later, when the student accounting 
system becomes sufficiently efficient to provide relevant 
and reliable data on terminees (tho^se who have terminated 
their enrollment from vocational programs), these two groups 
should be dropped from the follow-up population. 

File Preparation 

The next step is termed file preparation, as illustrated 
in Figure 1. In this step, as in the rest of this document, 
the "follow-up year" will refer to the school year for 
which terminees would be the subjects of a follow-up survey; 
and the "current year" will refer to the school year fol- 
lowing the "follow-up year." For example, if it is desired 
to follow-up the vocational program terminees from the school 
year ending May 1973, then the 1972-73 school year will be 
termed the "follow-up year" and the 1973-74 school year 
ending May 1974 will be termed the "current year." 

File preparation requires the following sequence 
(reference Figure 1) : 

1. From the "follow-up year" enrollment file, separate 
and load on a blank tape the records of all terminees 
as reported by the teachers. Non terminee records 
may be transferred to the "current year" enrollment 
file. Designate the file containing terminee 
records as the "follow-up file." 

2. Check the follow-up file for invalid or incomplete 
identification numbers and invalid or incorrect 
entries in the "name," "address," or the "program 
code" fields. Sort the defective records by school 
and by program and list them. 
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Figure 1 
FILE PREPARATION 




3. Send the lists to respective teachers or other 
school officials to make necessary corrections 
using school records. 

4. Make corrections in the follow-up file. Those 
records which can not be corrected should be 
expunged from the file and listed by program and 
by school. These are exceptional cases which 
will need special treatment. 

5. The follow-up file now is ready with correct names 
and addresses for the next phase of the follow-up 
routine. 

Questionnaire 

Figure 2 presents a follow-up questionnaire which is 
recommended as a data collection instrument. A prototype 
of the instrument was designed and developed by Center 
staff with active support and cooperation of many experts 
from different states. The instrumient was also pilot- tested, 
refined, and further field-tested in Kentucky. For details 
of the instrument development, see Appendix A. 

As can be seen from Figure 2, one section of the 
questionnaire has been designed to collect data about the jobs 
on which former students are placed, the number of hours they 
work and whether or not they are satisfied with their jobs. 

Data are also collected from (1) those former students who 
continue their education, (2) homemakers, and (3") others 
who are not otherwise classified by any of the above mentioned 
categories. Some evaluative data are also collected to add 
yet another dimension to the evaluation and planning of 
vocational education programs. 

Pre-Orientation of the Population 

One of the most important activities in these procedures 
relates to orienting the follow-up subjects to the instrument, 
especially to the importance of completing it correctly. 
During the pilot-test of the instrument, when a number of 
non- responding subjects were interviewed to find out the 
reasons for non-response, a majority of the interviewees 
(53.9 percent) did not respond because they "did not think it 
was important." A great many (61.5 percent) interviewees 
also suggested that they would have responded if the purpose 
of follow-up had been explained to them in greater detail. 
(Other details and results of these interviews are con- 
tained in Appendix A.) It is therefore crucial to explain 
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AN IN'TIAL FOLLOW-UP SURVEY OF FORMER VOCATIONAL STUDENTS 
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Dear Formir Vocitionil Studint: 

Your school and thi Dipirtmint of Education in your $titi in 
your stiti. Oni part of this work is to find out what you art doii 
tional training. 



Pliasa givf thi young paople of today and tomorrow thi 
You can hilp plan bittir sirvica for f utura studants by an 
bottom of this paga.) Whan you finish, plaas^Alilthis fori 

Evarything you say will ba kapt confidantial. 
your individual answirs or nama. 

Thank you vary much for your ha^^om us 




If your namS]li^^^l|^rSocial Security 
number is not corraS^Mabel, please 
make the necessary corrKli^s below. 






.ana 



her to improve tducetion in 
/ you feel about your voca- 




^your ideas based on your experience, 
questions. (See the instructions at the 
in the enclosed envelope. 



II be provided a total of the answers, but not 




ture students. 



Sincerely, 



INSTRUCTIONS 
1. If you MP serving 



aryi please check this box only and then return the questionnaire. IZI 



er questions iff section I, and all other sections that apply tc you. Example, if you are working 
pme and also iglplled in college, answer questions in sections 1,11, and III. 



3. SpkjRlLProvided 1 
offe^ 



le end of this questionnaire for your comments. Please express your opinions freely and 
Improvement of vocational programs in your state. 



"Vocational Program" in the questionnaire means a course or a sequence of courses in any one of the following 
eight areas. 



(a) Vocational Agriculture 

(b) Distributive Education 

(c) Health Education 



(d) Home Economics 

(e) Business and Offke Education 

(f) Technical Education 



(g) Trade and Industrial 
Education 

(h) Special Vocational 
Education 
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Figure 2 (con tinued) 
I. THIS SECTION FOR EVERYONE 



1. Was the vocational program identified on the address 
label your first, second, or third choice at the time you 
enrolled? (check one) 

12 First choice 

2 Q Second choice 

^ □ Third choice 

If it was not your first choice, why didn't you enroll in 
the program you wanted? 

1 Q Courses wen filled 

2 D It wu not offend at my school 

3 □ I didn't have the requirements 

^ n Other reasons: ^ - 



2. Were you ever placed on a waiting list to enroll in a 
vocational program of your choice? 

□ Yes 

□ No 



4. Are you planning to enroll for more vocational or technical 
education within the next two years? 

24 nves 

□ No 

If "yes," please specify the field (s) in which you want further 
vocational education (check all that apply) 

QVocational Agriculture Education 

D Distributive Education 

□ Health Education 
□Home Economics Education 

□ office or Business Education 
□Technical Education 

\ □Trade and Industrial Education 

x 

Please also specify the job or jobs for which you would like to 
be trained. 



3. Old you complete the vocational program identified on 
the address label? 

□Yes 

□ no 

If "No," pleait indketi the reason(s) for terminating 
the program btfort completion, (check all that apply) 

□ change of occupational objective 
□Work on a job 

□ Financial reasons 
□Marriage 

□ Oislike for the school 
□Transfer to another school district 
□Poor health 

□other (Specify). 



5. Please rate the following services of your school by checking 
one of the boxes in each can. 

Very 

Good Average Poor 



M(a) Vocational 

Instruction 

(b) Another 

Instruction 

(c) Vocational Shop 
or Laboratory 

(d) Guidance and 
Counseling 

(e) Job Placement 



!□ 2^ 3^ 



!□ 



2^ 
2D 

20 

2a 



»□ 
3D 

3^ 
3D 
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Figure 2 (continued) 
II. THIS SECTION FOR THOSE NOW WORKING ON A JOB 
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10. 
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11. 
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13. 



After leaving school, how much time did you spend 12. 
looking foryour first job? 

1 D Had the job before leaving school 
^ D Looked for two weeks or less 
^ D Looked for three to four weeks 

* n Looked for one to two months 

* n Looked for more thin two months 
Who helped you to get your first job? (check all that apply) 

D Vocational teacher 

D School job placement service 
D Other school personnel 
D State employment agency 

□ Private employment agency 
D Relatives and friends 

□ other (Specify) 

Is your pre lent job your first job after leaving school? 

□ Yes 

□ no 

Wiat is your present job? (Specify job title and a brief 
description. Examples: Secretary in a law office or 
Machinist of automobile parts.) 



14. 



15. 



16. 
17. 



How many hours per week do you usually work on your 
present job? (check one) 

^ □Lessthan 20 hours 
^ □ 20 to 24 hours 
" ^ □ 25 to 29 hours 
^ □ 30 to 34 hours 

^ □ 35 to 40 hours-Skip to question number 12 

^ □ More than 40 hour3-*Skip to question number 12 

If you are currently working less than 35 hours per week, 
please check the retson(s) for doing so. 

□ Cannot find fulhtime job 

□ Full'time work wHk less than 35 hours 

□ Temporary cut-bKk by the employer 18. 

□ Enrolled in school or college 

□ Do not want full'time work 

□ Other (Specify) 
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Which one of the following statements best describes the relationship 
between your present job and the training program on the address 
label? (check one) 

1 □ Job same for which trained 
^ □ Job highly related to the training 
^ □ Job only slightly related to the training 
^ □ Job not related to the training 
Have you moved away from where you were trained? (check one) 

□ Yes 

□ no 

If "Yes," how far did you move? 

^ □ Lessthan 100 mileswithin the state 

^ □More than 100 mileswithin the state 

^ □ Out of tlie state 
How far is your place of work from your present residence? (check one) 

1 □ 0-10 miles 

2 □ 11-25 miles 
^ □ 26-50 miles 

* □ More than 50 miles 

What is the wage rate of your present job excluding overtime? 
^ □$1.59 or less per hour 

2 □ $1.60 to $2.00 per hour 
^ □$2.C1 to $2.50 per hour 
^ □ $2.51 to $3.50 per hour 
» □$3.51to$4.50p3rhour 
« □ $4.51 to $5.50 per hour 

□ $5.51 or more per hour 

What are your average overtime earnings per wtek on this 
job? 

How do you feel about your present job? (check one) 

^ □Like it very much 

^ □like it somewhat 

^ □Neither like nor dislike it 

Dislike it somewhat 

^ □ Dislike it very much 

How ofttn on your present job do you use the knowledge and 

skill enquired from your training? 

^ □ Frequently (use most of the time) 
^ □ Occasionally (use only some time) 

3 □ Seldom (use only rertly) 

* □ Never (never use it) 

11 



Figure 2 (continued) 
III. THIS SECTION FOR THOSE NOW ENROLLED IN SCHOOL OR COLLEGE 



1^. Whac IS the type of institution in which you are presently 
enrolled? (check one) 
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^ D Vocational High School 
2 □ Public High School 
^ □ Two year Junior or Community College 
^ D Public post-secondary technical institute 
* D Four year College or Universit/ 
® n Private Vocational or Business School 
□ Other (Specify) 



20. How related is your present educationi! program to the vocational 
course identified on the address label? 

Same field 
2 □ Highly related 
^ DOnly slightly related 

Not related 



IV. THIS SECTION FOR THOSE WHO WERE ENROLLED IN HOME ECONOMICS 



21 . Do you consider your vocational program adequate 
preparation for homemaking? 

29 DYes 
□ No 

D Does not apply-am not a homemaker 

22. If you had the opportunity, which one of the following 
areas in Home Economics would you like to study? 
(check all that apply) 

^ Qpood management 

^ QCare and guidance of children 

^ Qciothing management and service 

* D Home and community services 

Other (Specify) 



23. How often do you use your home economics training in your 
work at home? 

^DFrtquenth/ (u» most of the time) 

^ □ Occasiorially (use only sometime) 

^□seldom (use only rarely) 

'< *D Never (never use it) 
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26. What have you done most recently to look for a job? 
(check all that apply) 



V. THIS SECTION FOR THOSE NOT NOW WORKING 

24. Are you looking for a job? 

□ Yes 

□ No 

25. If "Yes/' how long have you been looking for a job? 

^□less than one month 

More than one but less than two months 
^ □ More than two months 



^ □ ChKked with the school placement service 
^□chKked with a public or private employment agency 
3 QChKked directly with emjiloyers 
^ □ ChKked with frierrds or rttatrves 
5 □ Raced or answered newspaper ads 
6 □other (Specify) _ 



VI. YOUR COMMENTS AND/OR SUGGESTIONS 
Please use this space to give any additional comments and suggestions not covered by the ansvifers on this and the prKeding pages. 
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the purpose, need, and the importance o£ follow-up to the 
subjects* This will not only increase the rate of response 
to the questionnaire, but will also reduce the cost of 
conducting the survey. 

There are generally two ways of orienting the subjects. 
The more effective and less expensive of these two methods 
is orientation through the vocational teacher and/or the 
counselor. Needless to say, the appropriate time for this 
orientation is before the subjects leave the school or 
educational institution. Excellent results can be achieved 
with a one-to-two hour session with graducating seniors, 
which should include the following: 

1. The purpose, need, and importance of the follow-up 
survey data should be explained in detail. (It 
will be profitable for the state office to design 

an audio-visual package which can be used by teachers 
and/or counselors. This will reduce time spent in 
briefing and preparation of teachers and counselors 
and will also ensure uniformity of explanation 
without taking away the personal touch which only 
individual teachers and/or counselors can give.) 

2. A blank questionnaire should be shown to the 
students. (Or, it may be reduced to a number 
of transparencies which can be projected.) At 
this time the presenter should be willing to 
answer any specific questions regarding the in- 
strument . 

3. It is a good policy to circulate some of the re- 
ports from previous years among the students. 

It can be pointed out that after students com- 
lete and send back the questionnaires to the state 
office, their names are removed and are not used 
for making reports or analyzing data. The students 
may need to be reassured that the information they 
provide will be treated confidentially and that 
their names will not be revealed to teachers or 
other school officials. The only reason to retain 
names on the returned questionnaires is to identify 
and remind those who do not respond. 

This orientation session can be held at any time during 
the graduating senior year, but it will bring better results 
if it is held during the final two months. In either case 
a short reminder should be given to each graduate at the 
time of leaving school. 
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A similar process can be adopted for those who leave 
educational institutions before graduation (dropouts and/ 
or early completors) . A counselor or teacher can explain 
to them the purpose of follow-up and the importance of 
completing and returning the questionnaire promptly. Using 
this technique, the data collection can be completed faster, 
more efficiently, and at less cost inasmuch as it cuts down 
on the cost of reminders and other means of persuasion. 

Although orientation of students while they are still 
in school has certain advantages over orientation by mail, 
it may be more expedient and convenient to resort to mailed 
orientation because of organizational, personnel, or 
scheduling problems. 

If so, two weeks prior to the mailing of the follow-up 
questionnaire, a letter should be mailed to the subjects 
explaining the importance of the follow-up data and urging 
them to complete the questionnaire when it arrives. (A 
sample letter can be seen in Figure A-3.) 

Even when using this mail orientation method, it will be 
profitable in terms of better response rate to invite the 
help of teachers. For the sake of uniformity, the letter 
explaining the purpose and importance of follow-up should be 
drafted and printed by the state office. The schools 
may then be asked to mail it over respective teachers' signa- 
tures. For best results the letters may be typed individually, 
but in urban school districts this may not be very practica- 
ble. However, the following points should be noted: 

1. A letter is better than a postcard to explain in 
detail the purpose and importance of the follow- 
up survey. 

2. As far as possible there should be a personal touch 
to the letter by printing it on school stationery,: 
using the first name of each student in addressing 
the letter (instead of a generic address such as 
"Dear Friend,") and/or hand-signing every letter 

by the teacher. 

3. The subjects should be reassured as to the con- 
fidentiality of their responses. 

Mailing Sequence 

In cases where the subjects have not been oriented to 
the questionnaire before leaving school, the first piece 
of mail should be a letter (termed pre-letter) which explains 
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the purpose and need (see Figure A-3). A pre-letter is also 
recommended if the period between the in-school orientation 
and the follow-up is more than six months • In such a case 
this letter should serve as a reminder. As explained above, 
this letter should be signed by the respective teacher and/or 
the principal of the school if possible. 

After a period of two to three weeks from the mailing 
of the pre-letter, the questionnaire should be mailed to each 
subject. A forwarding letter, generally printed on the 
first page of the questionnaire (Figure 2) , and a postage- 
paid pre-addressed return envelope should be enclosed with 
the questionnaire for the convenience of the subjects.* 

A period of two to three weeks, should be allowed to 
elapse before the non-respondents among the subjects are 
reminded. The questionnaires returned during the interval 
should be carefully monitored by checking them off the 
mailing list. This process is important for the sake of 
economy and efficiency. All those who have not responded by 
the end of two weeks should be mailed a reminder requesting 
a quick response. (A sample reminder letter can be seen 
in Fig A-4.) It is generally profitable to enclose another 
questionnaire and reply envelope to make it convenient for 
the subjects to respond. This may create a problem of 
duplicate responses from the same person but a simple com- 
puter routine can remedy this situation adequately. 

If the subjects have been well oriented to the follow- 
up, the response rate to the questionnaire should reach 
70 percent by the end of two to three weeks after the mailing 
of a reminder. In this case, data collection can be stopped 
and the data are ready for the next step in the procedure. 
However, during early days of implementation, when pre- 
orientation is either non-existent or is by mail only, it 
may be necessary to remind the subjects a second time. 
This will also be true if higher than 70 percent response 
is considered necessary. A continued monitoring of the 
returns is important in order to mail a second reminder 
which may be mailed about three weeks after the first reminder 



*Some researchers have found that postage stamps affixed 
on a pre-addressed envelope brings better returns as compared 
with those using business-reply mailers. This technique, 
however, is of limited use for a comprehensive follow-up 
survey because the large population makes it impractical to 
stick thousands .of stamps on the envelopes. Wherever a 
smaller population is followed up, this technique of in- 
creasing responses may be used. 



15 



letter. This second reminder may be a letter or a post- 
card, signed once again by the respective teachers. Three 
weeks after the mailing of the second reminder, the completed 
questionnaires returned by the subjects should be ready for 
data processing and analysis. 

Sequence of Events 

In order to collect sufficient follow-up data in time for 
federal reports, the following sequence of evfents should be 
followed (shown in Figure 3). 

May 1 to May 31 - -Orientation sessions should be held in 
every school where vocational programs are offered. The 
purpose and uses of the follow-up survey should be explained, 
a follow-up questionnaire should be shown to the graduating 
classes, and students* questions should be answered in such 
a manner as to asfmre them of the survey's importance. 
Students may also be requested to inform their teachers 
of any change in their address after graduation. 

Those who leave school before ^graduation should be 
interviewed by their counselors or teachers to orient them 
to the follow-up and the questionnaire. 

June 1 through June 50-- A termination report on every 
student should be submitted by th,e teacher to the state 
office. 

September 1 to September 30 - -New enrollment data should 
be collected at the beginning of the school year and pooled 
in the state office to compile new enrollment listings by 
program and by school. 

September 16 to October 15 - -As the enrollment data 
start coming into the state office, follow-up file pre- 
paration should be initiated. This task should end by 
October 15. 

October 16 to October 31- -Follow-up questionnaires 
should be mailed during this period . 

November 8 to November 21 --During this period, a list 
of non^respondents should be generated and a reminder sent 
to each with a new questionnaire. 

December 1 to December 31 --During this period. Federal 
and other reports should be generated using an appropriate 
computer program. Also during the same period "bias check" 
procedures should be completed to check the validity of 
results. 
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January 1 to A^ril 30 --Results should be disseminated 
during this period tor use in the planning and evaluation 
of programs* Available data can also be used for special pur- 
poses as they are needed* 

Important Note : It should be remembered that the sequence 
outlined above is only a suggested or an exemplary one. A 
similar sequence with different time periods can be charted 
to meet different reporting deadlines. However, some of the 
events do not change from year to year and thus are fixed for 
the purpose of follow-up. For example, orientation cannot be 
shifted to a later date since most U.S. schools close by the 
end of May. Similarly, the best time to collect enrollment 
data is during the month of September when schools reopen and 
students register. 

On the other hand, the system is quite flexible and the 
sequence can be changed to suit a different reporting schedule 
or other user requirements. This may be especially true 
during early stages of implementation when the staff may require 
more time than the sequence allows to gain experience and 
practice. 

Cost : On the basis of 10,000 subjects, the cost of con- 
ducting a follow-up survey as described here has been approx- 
imately 60^ per subject. This cost, however, does not include 
the cost of personnel which should be figured separately and 
will vary from state to state or from user to user. 

Interpretation of Data 

Generalizability of the data to the total population is 
one of the most important elements of data interpretation. 
Two factors contribute to the level of generalizability . They 
are: 

1. High response rate 

2. Lack of bias in response due to non-respondents 

Methods to increase response rate were described in the pre- 
ceding chapter. Using those methods a high response rate can 
.be reasonably assured. 

To check possible bias due to non-response, it is 
necessary to randomly draw a small sample from those who 
do not respond to the follow-up questionnaire in spite of 
various reminders. A ten percent sample is generally con- 
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sidered adequate for this purpose, though a smaller sample 
will suffice in most cases, especially if a trend in res- 
ponses has already been established by analyzing and comparing 
responses before and after the reminders. 

Hiring a team of interviewers to contact the persons 
in the sample is most effective but fairly costly. The 
alternative, which is used by many states, is to request 
teachers, counselors, and others to try to contact (personally 
or by telephone) the selected subjects and persuade them to 
send in their completed questionnaires. (Often, the best 
place to start is the home town of the former student 
even when he has moved to a different city or state. His 
former friends, relatives, acquaintances, the local post- 
master, or the telephone company can often furnish his 
whereabouts.) This method can be very effective because 
the work load is distributed among many, thus making it 
easier to handle. 

Responses from the questionnaires received from this 
sample should be compared with those received earlier. One 
method of comparison used during the development of this 
system is described in Appendix A. Other methods can be 
found in any standard statistics textbook. 

No hard and fast rules are recommended for the in- 
terpretation of the data. The following two ways that the 
follow-up data may be interpreted are described as examples 
only. The first relates to the evaluation of programs 
against a set of measurable objectives. For example, an 
objective of a welding program could be to place 70 percent 
of its graduates on jobs related to welding. Given this 
objective, the follow-up data provides a measure of success 
of each welding program in the state. Successful programs 
will have graduates working on related jobs in excess of 
70 percent whereas those programs placing less than 70 
percent of the graduates on related jobs will be rated 
as failures in achieving that particular objective. 

Similar objectives can be set for differeit vocational 
programs in terms of graduation rate, job satisfaction, 
wage rate, etc., data on which are provided by follow-up 
tables . 

A second method of data interpretation is an evaluation 
of vocational programs by comparison with other similar 
programs. Thus, if the terminees £rom a given welding 
program earn a higher average wage than those from other 
welding programs within the region or within the state, 
the program might be considered the best based on wages. 
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Comparisons can be made using various other criteria such 
as job satisfaction, job placement, graduation rate, etc. 

Comparative analysis can not only be made between 
similar programs, but also between programs from different 
program area service divisions. A nurses aide program can 
be compared with an auto mechanics program provided such 
a comparison is meaningful to the analyst or the decision 
maker. 
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Appendix A 

Development Phase of Initial Follow- 
o£ Former Vocational Students 
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Appendix A 



Introduction . 

The following procedures relate to the development' phase 
of follow-up activities carried out in Kentucky, which 
consisted of the following: 

1. Development of the instrument 

2. Data collection 

3. Development of a software package for processing and 
analyzing the follow-up data. 

Instrument . A basic follow-iip instrument (questionnaire) 
was developed at CVE drawing, wherever appropriate, from 
instruments developed in other states. This instrument was 
later modified and refined after consultation with state and 
local vocational education administrators in Kentucky, 
selected experts at CVE, and some nationally known experts 
on the subject. The final draft was sent for approval to the 
Office of Education, Washington, D.C., and the Committee for 
Protection of Human Subjects. The instrument can be seen in 
Figure A-1. 

Pretesting of the Instrument . A pretest of the draft 
instrument was held to determine comprehension and readabil- 
ity. A group of twenty, composed of high school students 
and graduates, was asked to complete the instrument without 
any guidance, help, or explanation. Eight of the group 
experienced no difficulty in understanding the questions, 
whereas the rest expressed some problem with five of the 
thirty-one questions. Those five questions were suitably 
amended to make them easier to understand and interpret. 

Sample for the Pilot Test . A pilot test "of the follow-up 
instrument was held between April and September, 1972. The 
sample for the test consisted of 10,806 subjects, the back- 
ground data for which was supplied to CVE by the Kentucky 
Bureau of Vocational Education (KBVE) c This sample included 
7,432 secondary, 2,908 post-secondary, and 437 adult termin- 
ees who were defined a^ students terminating their vocational 
education during the school year 1970-1971. Geographically, 
the population represented fourteen of the fifteen planning 
regions in Kentucky and represented terminees from all area 
vocational schools and county extension ^;onters. The divi- 
sion of the sample by program service area was as follows: ^ 
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Form CiS'U 



Figure A-1 



SURVEY OF FORMER 
VOCATIONAL 
STUDENTS 



r 



L 



BUREAU OF VOCATIONAL EDUCATION 
DEPARTMENT OF EDUCATION 
FRANKFORT, KENTUCKY 



Dear Former Vocational Student: 

Your school is cooperating with the State Department of E^tfetSJn to conduct a statewide survey of 



all former students of vocational education. The p«rpos 
of vocational education in Kentucky so that you can provi 

Since you are the most important source Q^iatorma 
is essential for the success of this plan. Pie, 
postage-paid envelope to return it as soon as 




The information and opinions you give will be 
only an analysis that will not identify you a^ ind 

Thank you very much for your help 




is^sunrey is to improve the quality and level 
ervice to the students who follow you. 

is improvement process, your cooperation 
questionnaire and use the pre-addressed, 

strictly confidential. Your school will be given 




iiiuiir% jww v^tj iiiu\^ll lUI JfWUI I ^^^^^^^^ ^^^^^ 

Assistant Superintendent for Vocational 



^#incere 



ly yoursfy 



If your name and addr 
NEW NAME: 





lierinter 



Suiierintendent of Public Instructii 



istructi^jri 



INSTRUCTIONS: 

1. Most of the questions require^ 

2 "Vocational Program" in thei 
vocational school, o^^ge, oVjMittMfy. 

3 If you were ever^^Bfmd in Home Economics, answer questions 21, 22, and 23 in addition to the section most 
appropriate to yf ^ 



mark in one of the boxes provided for your response, 
aire refers to the last vocational course you attended in high school, area 



ove is not correct, please supply the following information: 



Ust Name 



Initials 



NEW ADDRESS: 



Number and Street or Rural Route 



ERIC 



City, Town, or Post Offica 



State 



Zip Coda 
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Figure A-1 (continued) 



ABOUT YOU 

1. Was the vocational program identified on the address 
u label your first, second, or third choice? (check one) 

First choice 
Second choice 
Third choice 
No choice available 

2. Were you ever placed on a waiting list to enroll in a 
vocational program of your choice? 

Yes 
2Q No 

3. Did you complete the vocational program identified 
on the address label? 

Yes 
No 

If "No," please indicate the most likely reason for 
terminating the program before completion, 
(check one) 

Change of occupational objective 

Financial reasons 

Dislike for the school 

Transfer to another school district 

Poor health 

«□ Other (Specify) 



4 Do you consider the vocational education received 
15 in this program adequate for your present activity? 

Yes 
No 

5, Are you interested in more vocational or technical 
education? 
Yes 
No 

If "Yes," please specify the field in which you want 
further vocational education, (check one) 
iQVocational Agriculture 

Distributive Education 

Health Education 

Home Economics Education 
5[— ] Office or Business Education 

Technical Education 

Trade and Industrial Education 

Please also specify the job or jobs for which you 
u would like to be trained. 



6. Please rate the following by checking one of the 
27 boxes in each case. 

Excellent Good Average Poor 

12 3 4 

(a) Your school □ □ □ □ 

(b) Your training □ □ □ □ 
progriSfii 

(c) Instructor □ □ □ □ 

(d) Shop or □ □ □ □ 
laboratory 

(e) Guidance and Q Q Q D 
counseling 
service 
provided by 
the school 

(f) Job placement □ □ □ □ 
services 
provided by 
the school 

7. Which of the following statements best describes 
33 ^our current status? (check one) 

iQ Working on a job— Full-time or Part-time 
(Go on to question number 8) 
Enrolled in school and also working 
(Go on to question number 8) 
Homemaker networking on a job 
(Skip to question number 21) 
Enrolled full-time in a school, college, or 
university (Skip to question number 24) 
5Q Networking, but looking for a job 
(Skip to question number 27) 
Serving with the Military 
(Skip to question number 30) 
7 Q Not working, and not looking for a job 
(Skip to question number 30) 

WORKING ON A JOB 

8. How long after leaving the vocational program did 
you get your present job? (check one) 
1 Q Had the job before leaving the school 

Within two weeks after leaving school 
3 Q Between two weeks and one month 
More than one month 

9. Who helped you most to get your present job? 
(check one) 

Vocational teacher 
School job placement service 
Private employment agency 
Relative and friends 
None of the above 



Figure A-1 (continued) 



10. What »s your present job? (Specify job title and a brief 
description. Examples: Secretary in a law office or 
Machinist of automobile parts.) 



11. How many hours per week do you usually work on 
your present job? (check one) 

Less than 20 hours 
2 □ 20 to 24 hours 
3 □ 25 to 29 hours 

30 to 34 hours 

35 to 40 hours— Skip to question number 13 
More than 40 hours— Skip to question number 13 

12. If you are currently working less than 35 hours per 
week, why are you doing so? (check one) 

Cannot find full-time job 
Full-time work week less than 35 hours 
Temporary cut-back by the employer 
Enrolled in school or college 
5Q«i3b not want full-time work 

Other (Specify) 



13. Which one of the following statements best describes 
the relationship between your present job and the 
training program on the address label? (check one) 

Job same for which trained 
Job highly related to the training 
Job somewhat related to the training 
Job not related to the training 

14. Is your present job your first job after leaving school? 

Yes 
2Q No 

15. Did you have to move to a place away from where 
you were trained? (check one) 

Yes 
No 

If "Yes," how far did you have to move? 
1 Q Less than 100 miles within the state 
More than 100 miles within the state 
3Q Out of the state 

16. How far is your place of work from present residence? 
(check one) 



0-10 miles 
11-25 miles 
26-50 miles 
5M00 miles 

CD?/-Q ^^^^ "^"^^ 



17. What is the liourly wage rate of your present job? 
(check one) 

52 '^^^ ^^^^ 

$L60 to $2.50 per hour 
$2.51 to $3.50 per hour 
$3.51 to $4.50 per hour 
$4.51 to $5.50 per hour 
«□ $5.51 to $7.00 per hour 
$7.01 to $10.00 per hour 
$10.01 or more per hour 

18. If you are not paid an hourly wage, what is your 
monthly income before taxes? (check one) . 

Less than $250 
$250 to $349 
$350 to $449 
$450 to $599 
$600 to $799 
6 □ $800 to $999 
$1,000 or more 

19. How do you feel about your present job? (check one) 
Like it very much 

2Q Like it somewhat 

Neither like nor dislike it 
*Q Dislike it somewhat 
Dislike it very much 

20. How often on your present job do you use the knowl- 
edge and skill acquired from your training? 

Frequently 
2Q Occasionally 
3Q Seldom 
*Q Never 

HOMEMAKING 

(For homemakers and former students of home economics 
only. Others please skip to question number 24.) 

21. Do you consider your vocational program adequate 
preparation for homemaking? 

Yes 
2Q No ' 

3Q Did not study Home Economics in high school 
Does not apply— am not a homemaker 

22. If you get another chance, which one of the following 
areas in Home Economics would you like to study? 
(check one) 

Food management 
2Q Care and guidance of children 
3Q Clothing management and service 
*Q Home and community services 

Other (Specify) — 
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Figure A-1 

23. How often do you use your training in home economics 
In your work at home? 

Many times daily 
2Q] At least once daily 
3Q] Seldom 

Does not apply 

CONTINUING EDUCATION 

24. Are you currently enrolled in an educational 
institution? 

Yes (Go on to question number 25). 
No (Skip to question number 30). 

25. What is the type of instiution in which you are 
presently enrolled? (check one) 

Public High School 
2Q] Two year Junior or Community College 

Technical Institute 
*n Fouryear College or University 

Private Vocational School 

Private Business School 

Other (Specify) 

26. How related is your^ present educational program to 
the vocational course identified on the address label? 
(check one) 

Same field 
Highly related 
3Q] Somewhat related 
Not related 

(Skip to question number 30). 



(continued) 

NOT WORKING ON A JOB 

27. How long have you been unemployed? 
^ Less than one month. 

More than one but less than two months 
3|--j More than two months 

28. What have you done most recently to look for a job? 
(check one) 

Checked with the school placement service 
Checked with a public or private employment 
agency 

Checked directly with employers 

Checked with friends and relatives 

Placed or answered newspaper ads 

Other (Specify) 

29. Please list three job titles for which you consider 
yourself qualified. — 

64 



30. Please print your social security number for future 
reference 

□ □□ □□ □□□□ 

31. The following space has been provided for your frank 
comments about your school and your program. The 
information will be used as guidelines for program 
improvement only and your comments will be con- 
sidered STRICTLY CONFIDENTIAL The space may 
also be used tc explain some of the answers given 
above. 



Vo a "t* T OTi a 1 A CTT i fl 1 1 t*l 1 TP Pri 1 1 f fl t* 1 nn 




274 


(1. 






1 


t \J *J ^ 




J- / 






779 


(7 . 


21 


Hnmp FrnrmTnir^ Ftliirfj t* i nn 




659 






Riminp^^ and Office Education 


1 


,980 


(18* 


3) 


Technical Education 




207 


(1- 


9) 


Trade and Industrial Education 


4 


,981 


(46. 


1) 


Special Programs 




285 


(2. 


6) 


Total 


10 


,798** 







^Figures in parentheses represent percentages that may not 
total 100 due to rounding errors. 
**Eight records had wrong or missing program codes. 

The background data for this terminee sample were supplied 
to CVE on a computer tape from which a computer tape file was 
created for every member of the sample. 

Data Collection . The tape file was used at CVE to print 
address labels to mail the questionnaires. The format of the 
labels was as follows: 

NAME OF THE PROGRAM SOCIAL SECURITY NUMBER 

NAME OF THE STUDENT 

ADDRESS 

CITY, STATE, ZIP CODE 

An alphabetic list of names and social security numbers 
of all terminees was also computer-printed with the format 
shown in Figure A- 2 to form a register that was used to monitor 
the mailing of questionnaires and subsequent reminders wher- 
ever necessary. Complete records of the dates of mailings 
and responses were kept by KBVE personnel. CVE worked closely 
with the Kentucky state staff to help work out record keeping 
procedures . 

A follow-up questionnaire was mailed to every terminee in 
the sample. After a waiting period of one week, all non- 
respondents »were mailed a reminder and a fresh questionnaire. 
After an additional wait of three weeks, another reminder and 
questionnaire were mailed to those who still had not responded. 
A total of 1,069 questionnaires were returned by the post office 
as undeliverable. The results of this data collection effort 
are reported in Table A-1. 
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Figure A-2 



Format of the Register Used for Monitoring 
of Follow-up Returns 



NAMK 


Social 
Security 
Number 


DATES 


1st 
Mail 


Return 


2nd 
Mail 


Return 


3rd 
Mail 


Return 


Phone 


Last Initials 





















TABLE A- I 

Response Rate to Follow-up Questionnaire 



Questionnaire^ 


Numbers 




Percent 


Total mailed 


10,806 




100.0 


Returned by post office 


1,069 




9.9 


Responses to 1st fuaii Out 


2,915 




27.0 


Responses to 2nd mail out 


1,067 




9.9 


Responses to 5rd mail out 


1,105 




10.2 


Others* 


102 




1.0 


Total Usable Responses 


5,189 




48.0 



^Responses received as a result of telephone contacts/ 
interviews with the respondents. 
**The research design outlined in this section was suggested 
and initiated by KBVE personnel including Floyd McKinney, 
Billy Vice, and Charles 0. Neel. CVE supported the effort 
and provided the data analysis. 
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Mailing Sequence > Items were mailed to the terminee 
sample according to the following sequence: 



1. A pre-letter post card, one week prior to the mailing 
of the follow-up questionnaire, to give advance notice 
(Figure A-S) . 

2. The follow-up questionnaire. 

3. A post card expressing appreciation to those responding 
to the follow-up questionnaire, or a reminder, mailed 
one week after mailing the follow-up questionnaire, 
printed in a different color for easy handling and 
recognition. 

4. Another follow-up questionnaire, printed in a dif- 
ferent color for easy handling and recognition. 

In order to empirically test the relative efficacy of 
various alternative sequences of mailing pre-letters, question- 
naires, and reminders, the sample v/as randomly divided into 
twenty groups. The groups were again randomly selected for 
the four treatments. 

Treatment 1: Advance notice, a questionnaire, and (if 
no response) another questionnaire 

Treatment 2: Advance notice, a questionnaire, a reminder, 
and (if no response) another questionnaire 

Treatment 3: A questionnaire, and (if no response) a 
reminder and another questionnaire 

Treatment 4: A questionnaire, and (if no response) 
another questionnaire 

The selection of groups for treatment was random and is 
shown in Table A-II. As can be seen from the table, every 
member of the population received at least two questionnaires, 
and the groups to which nothing else was mailed (i.e., pre- 
letter or reminder) served as the control groups. Responses 
to the follow-up questionnaire were carefully monitored for 
every group. 

From the results presented in Table A-III, it can be 
seen that the response rate was highest with treatment T^ and 
the lowest rate was with treatment T4, which was the control 
group. A significatn difference in the rate of response was 
found as a result of treatments. Since the sequence of pre- 
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Figure A-3 
ERE-LESTER 



April Jj 1972 

Dear Former Vocational Education Student: 

In a few days you will receive a questionnaire 
requesting information regarding the training you 
received as a vocational education student. This 
information will be very helpful in improving programs 
of vocational education. We hope you will be able to 
respond quickly to the questionnaire. 
WE PROMISE^ the information you give us will he Btriotly 
confidential I 





AsgiSt^t Superinte;idenJ( for Vocational Education 



Superintendent ofPubHc InsS uction 



: Invdruc 



C. 0. Neel, Coordinator 
State Program Evaluation 

Bureau of Vocational Education MlBf^^iiy';. Vl^ 

Fifth. Floor, State Office Building ' [ ) 

Frankfort, Kentucky 40601 ' Ktjv^'- 'J- 

U&I>ostageO^ 
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TABLE A- I I 
Nomination of Groups for Mailings 



Group 




TREATMENT BY 


FOUR MAILINGS 




Card (Ml) 


Instrument (M2) 




luo L rumen L i imh ) 


1 


X 


X 




X 


2 


X 


X 


X 


X 






Y 

A, 


X 


V 
^\ 


4 




X 




X 


5 


X 


A 




X 


6 


X 


Y 

A, 


X 


X 


7 




Y 

A, 


X 


X 


Q 
o 




Y 
A 




X 


9 


X 


X 




X 


10 


X 


X 


X 


X 


11 




X 


X 


X 


12 




X 




X 


13 


X 


X 




X 


14 


X 


X 


X 


X 


15 




X 


X 


X 


16 




X 




X 


17 


X 


X 




X 


18 


X 


X 


X 


X 


19 




X 


X 


X 


20 




X 




X 


Total 


To 


21 







letter-questionnaire-reminder-questionnaire represented by 
treatment T2 brought maximum response from former students, 
it was recommended for use in the second and subsequent cycles 
of follow-up. 

Bias Check Due to Non-Response . A randomly selected 
sample of sixty was drawn from those who did not respond to 
mailed questionnaires and reminders. This sample was later 
amended to exclude the names of persons listed residing out- 
side the state and some others who were otherwise hard to 
locate. The latter included those whose name and address were 
incorrect on the computer tape and also those who had moved 
leaving no forwarding address. All of the above were excluded 
from the sample because of the high cost of interviewing out- 
of-state persons and of finding the hard-to-locate. Other 
randomly selected names were added to the sample to bring it 
to a total of sixty. . ^ . 
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An experienced interviewer arranged to visit with every one 
of the persons selected in this sample. One o£ the persons 
refused to be interviewed or to give any assistance in data 
collection, and the parents of nine others informed the inter- 
viewer that the subjects were in military service and thus 
unavailable for interview. Each remaining subject was requested 
to complete a follow-up questionnaire provided by the inter- 
viewer who was instructed not to offer any help or explanation 
to any of the questions. After the completion of this task the 
interviewer asked a few simple questions directed at probing 
the reasons for non-response. Data from the interview schedules 
are reported in Table A-IV. 

The most frequently reported reason for non-response was 
"Did not think it was important," which was also reflected in 
suggestions for improvement. A majority (61.5 percent) of persons 
wanted more and better effort by those conducting follow-up 
to explain the purpose of the questionnaire to the subjects. 
The results of this survey were used to improve the next follow- 
up cycle by explaining the purposes of follow-up in a pre- 
letter to be mailed prior to the questionnaire mail-out. 

Data from the follow-up questionnaires completed at the time 
of interviews were tabulated and compared with the responses 
received earlier. Whereas both sets of data followed a similar 
pattern of response, on most of the questions the data from the 
interview sample were inadequate for statistical analyses. 
Therefore, only eight important variables were selected to 
statistically compare data from the two sets. Result of this 
comparison is reported in Table A-V. 

A significant difference at the 0.05 level was found only 
in one of the eight variables: "Relatedness of Jobs to Programs." 
For all other variables there was no significant difference 
between the two sets of responses. It can be concluded from 
this analysis that in the total responses, the persons placed 
on jobs related to training programs seem to be over-represented. 
However, no adjustment to data was made because the difference 
due to this factor was slight. Except on that one factor, the 
responses already received did represent the responses of the 
total population of terminees, and results of the analyses could 
be generalized for the population surveyed. 

Validation of Information . A randomly drawn sample of 
fifty persons was selected from those respondents to the follow- 
up questionnaire whose known residence was within the state. 
Financial and other resource constraints prohibited the inter- 
view of out-of-state respondents, who represented a total of 
about 6 percent of the total responses. 
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Table A- IV 

Aggregation of Data from Interviews 
of Non-Respondents to the Follow-up Questionnaire 



Total number of interviews - 52 
1. Did you receive the questionnaire in the mail? 



Yes 
No 



51 (98.1)* 
1 ( 1.9) 



2. If "yes," did you complete and mail it? 



Yes 
No 



6 (11.5) 
46 (88.5) 



If "No," please check one or more of the following 
reasons for not completing the questionnaire 

(a) Lost the questionnaire 

(b) Other family member threw it away 

(c) I did not think it was important 

(d) Away from home most of the time 

(e) Waiting to find a job 

(f) Thought it was to be completed 
if working on a job 

(g) Questionnaire too long 

(h) Did not understand the questions 

(i) Objection to some of the questions 
(i) Did not like the program 

(k) Did not like vocational teacher 

(1) Hid not like the school 

(m) Doubt in confidentiality 

(n) Did not know the answers 

(o) Did not want to give information 

(p) No reason at all 

(q) Other 

Too busy, no time 

Thought it was mailed 

Did not receive 

Lazy 

Thought did not apply 
(still in school) 



3 
3 
28 
10 



7 
20 
11 
3 
1 
2 
3 



( 5.8) 
( 5.8) 
(53.9) 
(19.2) 



1 ( 1.9) 



1 ( 1.9) 



( 3.8) 
( 1.9) 
( 1.9) 



(13.5) 
(38.5) 
(21.2) 
( 5.8) 
( 1.9) 
( 3.8) 
( 5.8) 



4. What could we have done to prompt better returns? 



(a) 
(b) 
(c) 
(d) 



(e) 



Pre-warning by teacher 

Discussion of importance while still in school 
Letter from teacher 

Better explanation of the purpose in the 
letter on the questionnaire 
Slick paper 

Different color of paper 
Other 

Personal note or interviewers 
Stress more importance 
Specific time for return 



*Numbers in parentheses represent percentages 



9 
5 
3 
1 



( 1.9) 
(15.4) 



32 (61.5) 



(17.3) 
( 9.6) 
( 5.8) 
( 0.9) 
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Table A-V^ 



Comparison Analysis o£ Respondents and 
Non-Respondents to the Follow-up Questionnaire 



.Von-Resp. 
\-50 



Resp. 



Chi Square 
Signi f icant 
at Level : 



1. Variables Related to Job 
(a) Status: 

Working 
Not working 



(b) Relatedness of job to program: 

Same Occupation 
Highly related 
Somewhat related 
Not related 



(c) Vages.per hour: 



Less than $1.60 
Si. 61 to $2.50 
S2.51 to $3.50 
More than $3.50 



(d) Full/part time: 



Full time 
Part tine 



Variables Related to School 

(a) Completion of programs: 

Gr^aduates 
Dropouts 

(b) Current Status: 

Enrolled in school 
Not enrolled 



(c) Relatedness of present education 
to vocational program: 

Same field 
Highly related 
Somewhat related 
Not related 



(d) Keed more vocational education: 

Yes 

No 



35 (59.3) 
24 (40 . 7) 



3 ( 8.6) 

3 ( 8.6) 

7 (20.0) 

22 (62.9) 



5 (14.3) 
19 (54.3) 

6 (17.1) 
5 (14.3) 



4 (11.4) 
31 (88.6) 



30 (60.0) 
20 (40 .0) 



14 (28.0) 
36 (72.0) 



6 (42.9) 

2 (14.3) 

3 (21.4) 
3 (21.4) 



32 (64.0) 
18 (36.0) 



2409 (60.8) 
1872 (39.2) 



549 (19.4) 

446 (15.8) 

680 (24. i) 

1149 (40.7) 



229 ( 9.0) 

1487 (58.7) 

520 (20.5) 

299 (11.8) 



457 (16.7) 
2364 (83.3) 



2974 (61.6) 
1069 (38.4) 



1031 (30.1) 
2393 (69.9) 



313 (26.7) 
118 (10.1) 
218 (18.6) 
524 (44.7) 



2842 (59.4) 
1939 (40.6) 



o.ao 



0.01 



0.50 



0.30 



0.80 



0.70 



0.30 



0.70 



Figures in parentheses represent percentages that may not total 100 due 
to errors of rounding. 
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An experienced and trained interviewer visited with every 
person (except one who had moved out-of-state after responding 
to the questionnaire) who was asked to complete a follow-up 
questionnaire supplied by the interviewer. After the completion 
of this task, each person, in the sample was asked a few simple 
questions by the interviewer to complete an interview schedule, 
which can be found in Figure A- 5. These questions were asked 
to find out if there was uniformity among the respondents with 
regard to the interpretation of selected questions. The data 
resulting from this interview schedule is given in Table A-VI. 

Revision of Instrument and Procedure for Next Cycle . 
Utilizing the experience gained and the findings of data 
analysis from the pilot test follow-up, certain changes were 
recommended in the procedures as follows: 

1. A detailed letter (instead of a post card) should be 
mailed to terminees about two weeks prior to the mail- 
ing of questionnaires. The purpose and objectives 

of follow-up should be explained in this letter and the 
recipient's cooperation should be requested. In later 
cycles, the objectives and procedures of follow-up 
should be explained to students before they terminate 
their vocational education. Teachers and counselors 
can acquaint the students with the questionnaire and 
allay their apprehensions about unauthorized use of 
data. This procedure should achieve a better response 
rate than a pre-letter. 

2. A two-week period should be allowed between the mailing 
of a pre-letter and the questionnaire. 

3. Cooperation of teachers, other school officials, and 
the community at large represented by parents and 
other social organizations should be sought through 
appropriate efforts. 

4. A reminder should be mailed to non-respondents no 
more than three weeks after the mailing of the initial 
questionnaire. 

5. A second questionnaire should be enclosed with the 
reminder. 

The instrument was also amended slightly to remove some 
of the problems encountered by a small number of respondents. 
Also, some changes were indicated for ease of computer pro- 
gramming and data analysis. A revised draft instrument was 
approved by KBVE officials, the U.S. Office of Education, and 
the Committee for the Protection of Human Subjects. 
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Figure A-4 
FOLLOW-UP LETTER 



April 28, 1972 

Dear Fonner Vocational Education Student: 

We deeply appreciate your participation in the 
^ Survey of Fonner Vocational Students. No one else 
can give us the insights and experience which 
you have — and which we ne^d. 

If your questionnaire is already ^ in. tha mail» thank 
you for your help. If not» could you nail it today? 




Assistant/ Suoerintendttt for Vocational Education 



Superintendent of Public Instruct 



ion 



CO. Neel, Coordinator 

State Program Evaluation ^,^^r!:3:>v ' 

Bureau o£ Vocational Education PaiiiKnTre \ - XPtiriot 

Fifth, Floor, State Office Puilding^ ; ) 

Frankfort, Kentucky 40601 \ * [ 
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Figure A- 5 

A Survey of Respondents to the 
Follow-up Questionnaire 



NAME SOCIAL SECURITY NO, 

DATE 



1, DID YOU UNDERSTAND EVERY QUESTION ON THE QUESTIONNAIRE? 

□ YES 

□ NO 

IF "NO/" PLEASE SPECIFY THE QUESTIONS WHICH YOU DID NOT 
FULLY UNDERSTAND, 



2. HOW DID YOU INTERPRET PRESENT ACTIVITY IN QUESTION NUMBER 
4? 

□ WORK ON YOUR JOB 

□ WORK RELATED TO SCHOOL 

□ WORK RELATED TO HOME 

□ WORK RELATED TO HOBBIES 

□ OThuK WORK (please SPECIFY . '. 

) 

3. ON QUESTION NUMBER 6 WHAT WAS THE BASIS OF YOUR RATINGS? 

□ REPUTATION OF THE SCHOOL 

[] PRINCIPAL OR OTHER OFFICIALS 

□ OTHER STUDENTS 
[[] ATHLETIC PROGRAM 
[] ACADEMIC PROGRAM 

□ OVERALL QUALITY OF SCHOOL 

□ OTHER (specify ^ 

48 
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Figure A-5 (cost.) 



^. please explain your response to questions 13/ 20/ 26/ and 
28. (only those that apply) 



5a on the question of wages did you report after/before-taxes- 
wages or salary? 

□ after taxes 

□ before taxes 

□ does not apply 

5b did you include over-time pay in the figure reported? 

□ YES 

□ NO 

□ DOES NOT APPLY 

6. HAVE YOU CHANGED YOUR OPINION ABOUT ANY OF YOUR ANSWERS 
TO THE QUESTIONS? 

□ YES 

□ NO 

IF "YES/" PLEASE EXPLAIN, 



7, CAN YOU SUGGEST ANY CHANGES IN THE QUESTIONNAIRE TO IMPROVE 
ITS CURITY? 
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Plgupe A-5 (cont.) 



WOULD YOU MIND IF WE CONTACT YOU IN THE FUTURE FOR FURTHER 
CURIFICATION OR FOR COLLECTING MORE INFORMATION? 

□ YES 

□ NO 
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Table A-VI 



Aggregation of Data from Interview of Respondents 
to the I-ollow-up Questionnaire 



1. Did yon undcrs t»ind every (jucstion on the 
ques t ionn*! i re** 


Yos 

No 


41 


83.7 
16.3 


2. How did you interpret present activity in 


Job 


39 


79 . 6 


question M? 


School 


3 


6.1 




Home 


1 


?.0' 




Hobbies 








Other 


6** 


12,3 



On que^ition #6 what was the basis of your Reputation of school 

riitinj!? Principal or other 



official 
Other students 
Athletic program 







Academic program 


2 


4.1 






Overall quality of 


44 


89.8 






school 










Other (Shop) 


2 


4. 1 






No answer 


J 




4. 


iNumbor commented showing tlie relatedness 




38 
\ 1 


77.6 




(or otherwise) o( their jobs 


No answer 


71 4 


5A. 


On the ijuestion of wages did you report after/ 


After taxes 


f) 


10.2 




before taxes -wages or salary? 


Before taxes 




.M 0 






Does not apply 


17 


.M.7 






No answer 


> 


h 1 


SB. 


Did you include over-time pay in tijo figure you 


Yes 


0 


12.3 




reported? 


No 


7 


14.3 






Does not apply 


30 


01.2 






No answr* r 


ft 


^ 


6. 


Have you changed your opinion .ibout any of your 


Yes 


1 


2.0 




answers to the questions? 


No 


46 


93.9 






No answer 


2 


4,1 


7. 


(i) No need to change (easy to interpret) 




16 


32.7 




(ii) Difficult for those enrolled in more 




1 


2.0 




than one program 








(iii) Question #3 should have more than two 




1 


2.0 




responses 










(iv) No suRgestion 






6^ 5 


8. 


Would you mind if wo contact )on in the future 


Yes 


1 


2.0 




for furthor clarification or tor collecting 


No 


45 


01.8 




more information' 


No iMWWor 




(> . 1 



*Qiicsti<in *IS (one) 
**hi'hatevor I am iloiiiu , iirrcnt l\ 
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Appendix B 
Sample Output and Interpretation 
Geherated During Development 
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APPENDIX B 



A number o£ tables were generated during the development 
phase, using a specially designed computer program package. 
Examples of table format are shown following the descriptions 
given below. 

Table B-I provides statistics on responses to each question 
in the questionnaire. In addition to total responses, compara- 
tive responses by sex, by program completions (graduates versus 
dropouts), and by employment (those working on jobs versus 
those not working) are printed, both in numbers and percent- 
ages. This table is designed for use by state officials involved 
in state-wide evaluation and planning. The State Advisory 
Council and selected state legislators are other potential 
users of this report. 

Table B-II presents state-wide and regional percentage 
responses to each of the questions in the questionnaire. The 
table should be useful for regional directors and/or coordina- 
tors. Along with Table B-X, this table can also be disseminated 
to local school administrators to allow comparisons between their 
schools and their region, neighboring regions, and the state. 
This should provide a useful tool^for self -evaluation. 

Table B-III represents the adjustments that need to be 
made to the projected terminee population of vocational programs 
so as to arrive at projected manpower supply figures. This is 
an important table for vocational education and manpower plan- 
ning. Since all those who complete training programs do not 
take jobs, it is important to estimate the number who will. 
This table helps to make such estimates for each program. 

The columns of the table are self-explanatory except for 
the last two. "Percent available" is defined as 

("total number working" + "unemployed but available" - 
"employed outmigrants")/ (total trained in the program). 

"Percent available related" is defined similarly, except that 
"total number working" is replaced by **number working related 
jobs." Outmigrants are defined as those who have taken employ- 
ment in another state. From these definitions, it is clear that 
"availability" is concerned with the state labor market. These 
two columns therefore provide an indication of the production 
of trained manpower for the state labor market by each program. 
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The target audience for this table are vocational education 
and manpower planners at the state level. 



One of the priorities in vocational education is focused 
on the service provided to the disadvantaged and the handicapped. 
This emphasis is further underscored by allocation of Federal 
funds specifically for vocational programs for the disadvantaged 
and the handicapped. It is appropriate, therefore, to assess 
how these vocational programs succeed in achieving their de- 
signed purpose. Tables B-IV through B-VI report follow-up data 
from the disadvantaged and/or handicapped subjects. Table B-IV 
represents "Current Occupational Status of the Disadvantaged/ 
Handicapped by Vocational Program Area;" Table B-V represents 
"Job Relatedness and Wages Earned by the Disadvantaged, by 
Vocational Program Area;" and Table B-VI represents "Reasons for 
Non-Completion and the Need for More Vocational Programs as 
Stated by the Disadvantaged/Handicapped, by the Type of Dis- 
advantage/Handicap . " 

Potential users of these three tables are state vocational 
planners of programs for the disadvantaged and handicapped and 
other State and Federal agencies involved in serving these two 
population groups. 

Next, a set of three tables (Tables B-VII through B-IX) 
has been designed for the program service area supervisors. 
These tables provide planning and evaluative data by vocational 
program. The tables are entitle,d "Analysis of Relationship 
Between Job and Training Course;" "Analysis of Relationship 
Between Educational Program and Vocational Course;" and "Anal- 
ysis of Unemployed But Available," respectively. Regional 
directors and local school administrators may be considered the 
primary audience for this set of tables since the data are 
broken down by individual school. 

Primary potential users of the next set of tables are the 
school officials, such as principals, local directors, teachers, 
counselors, and students. These tables include: 

Table B-X Responses to Questions by School 

Table B-XI Analysis of Non-Graduate Reasons for 
Termination 

Table B-XII Analysis of Jobs Related To Training 

Table B-XIII Analysis of Jobs Not Related to Training 

Table B-XIV Desire for Continuing Education as Related 
to Job Satisfaction 
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Table B-XV Analysis o£ Waiting List With Program 
Completion and Job Relation 



Table B-XVI Analysis o£ Ratings o£ Schools and Programs 

0£ this set, Table B-X is speci£ically designed £or use 
with Table B-II* Blank space is provided in Table B-II to £ill 
in by hand the responses reported in Table B-X. This composite 
table then provides a comparison o£ school responses to £ollow- 
up questions with those from all the planning regions as well 
as £rom the whole state* This is designed to be a tool £or 
program evaluation, planning, and vocational counseling at the 
local level* Data £rom this and other tables in this set are 
useful for planning and evaluation at the state and regional 
levels as well. 
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